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EXECUTIVE  SUMMARY 


Objective 

This  report  provides  a  utilization  assessment  of  the  Instructional  Support  Feature  (ISF) 
Guidelines  developed  by  the  Air  Force  Hunan  Resources  Laboratory,  Operations  Training  Division 
(AFHRL/OT).  Various  organizations  Involved  In  defining  training  requirements.  In  writing 
procurement  specifications,  and  In  system  development  were  contacted  to  determine  their 
utilization  of  the  ISF  Guidelines. 


Requirement 

A  research  and  development  (RAO)  effort  was  coordinated  by  AFHRL/OT  and  the  Aeronautical 
Systems  Division  (ASD),  with  Input  from  major  commands  (MAJCOMs),  to  provide  guidelines  for  use 
In  the  development  of  specifications  for  the  Instructional  component  of  aircrew  training  devices 
(ATOs).  The  formal  requirements  for  this  RAD  were  Request  for  Personnel  Research  (RPR)  76-20, 
Aircrew  Performance  Measurement  System,  and  Technology  Need  (TN)  ASD-0509-82-63,  Flight  Simulator 
Performance  Metrics. 


Products 


Two  AFHRL  technical  reports  (TRs)  resulted  from  this  RAD  effort,  of  which  the  ISF  guide  Is 
the  primary  product.  Documentation  of  the  RAD  Is  contained  In  AFHRL-TR -85-57(1),  Development  of 
Instructor  Support  Feature  Guidelines.  AFHRL-TR-85-57(II),  Instructor  Support  Feature  Guide¬ 
lines.  provides  procedures  for  analyzing  training  and  Instructor  support  requirements,  provides 
descriptions  of  ISFs,  and  describes  current  system  capabilities.  The  ISF  definition  portion  of 
the  guidelines  has  been  Incorporated  by  ASD  Into  the  following  military  standard:  MIL-S-8724, 
Military  Specification,  Simulator  Flight,  31  December  1985. 


Specifics 

The  ISF  guide  was  developed  under  work  unit  2359-01-06,  Performance  Measurement  System  for 
Aircrew  Training  Devices  (PE  63751F).  The  work  unit  was  approved  In  October  1983,  contracted 
with  Loglcon  Inc.  In  May  1984,  and  closed  out  as  completed  In  February  1986.  The  total  cost  of 
the  effort  was  $449,198,  of  which  ASD  provided  $100,000. 


Assessment  Results 

A  number  of  organizations  Involved  In  the  development  and  use  of  ATDs  were  surveyed  con* 
cernlng  utilization  of  the  ISF  Guidelines.  Although  some  Individuals  Indicated  that  the  guide 
has  limited  application,  a  majority  of  the  users  contacted  stressed  Its  usefulness  as  a  reference 
document,  particularly  for  definitions  and  applications  of  ISFs.  It  was  frequently  stated  that 
standardization  of  the  terminology  of  ATD  Instructional  systems  facilitates  the  determination  of 
what  features  need  to  be  Incorporated  In  ATDs  to  satisfy  the  training  requirements  specified  by 
operational  users.  The  ISF  Guidelines  are  only  a  part  of  the  total  documentation  needed  In  the 
development  and  acquisition  of  the  Instructional  component  of  an  ATD.  Considering  Its  usefulness 
as  a  reference  document  and  Its  1  ncorporatlon  Into  a  military  standard,  the  ISF  guide  1s  a 
success.  For  continued  effectiveness,  the  guidelines  need  to  be  continually  updated  to 
transition  lessons  learned  and  Incorporate  technology  advances  In  Instructional  systems. 


TABLE  OF  CONTENTS 


1.  INTRODUCTION  . 

II.  BACKGROUND  . 

III.  DEVELOPMENT  COST  . 

IV.  RESULTS . . 

Assessaent  Methodology  . 

Implementation  Assessaent . 

V.  RECOMMENDATIONS . 

VI.  CONCLUSIONS . 

APPENOIX  A:  INSTRUCTIONAL  SUPPORT  FEATURES  . 

APPENDIX  B:  RAD  PRODUCT  UTILIZATION  ASSESSMENT  SURVEY  .  .  . 

APPENDIX  C:  ORGANIZATIONS  PROVIDING  UTILIZATION  INFORMATION 

LIST  OF  FIGURES 

Figure 

1  Response  Frequencies  for  Utilization  Assessaent  Question 


INSTRUCTIONAL  SUPPORT  FEATURE  GUIDELINES: 
UTILIZATION  ASSESSMENT 


I.  INTRODUCTION 

Aircrew  Training  Devices  (ATOs)  have  two  Min  components:  the  simulation  system  and  the 
Instructional  system.  The  simulation  component  focuses  prlMrlly  on  Issues  regarding  the  degree 
of  fidelity  In  reproduction  of  the  aircraft  ana  flight  environment.  The  Instructional  component 
provides  those  capabilities  necessary  to  Mke  the  simulator  a  training  device,  through  Instruc- 
tlonal  support  features  (ISFs)  designed  to  enhance  the  training  process.  Through  an  Instructor/ 
operator  station  (I0S),  the  Instructor  can  use  the  ISFs  to  create,  control,  monitor,  and  record 
flight  training  missions.  Appendix  A  lists  available  ISFs,  organized  according  to  the  functions 
they  perform  and  In  the  order  they  would  most  likely  be  used.  Nhen  properly  designed  and 
utilized,  these  features  greatly  reduce  the  workload  Involved  In  conducting  training  exercises 
and  enable  the  ATD  Instructor  to  more  effectively  control  and  monitor  student  activities. 
However,  Instructional  systems  have  not  always  been  developed  according  to  the  users'  needs.  In 
early  attempts  to  build  ATDs,  there  had  frequently  been  an  emphasis  on  developing  comprehensive 
Instructional  systems  which  supported  every  aspect  of  Instruction,  Instead  of  training  devices 
designed  to  meet  specific  student  training  needs  and  Instructor  requirements.  This  has  resulted 
In  a  multiplicity  of  systems  that  are  not  responsive  to  the  Instructors'  needs  and  are  often 
difficult  to  use  and  understand. 

The  document  entitled  "Instructor  Support  Feature  Guidelines"  was  developed  to  enhance  the 
Instructional  support  of  ATDs;  l.e.,  to  provide  a  better  understanding  of  ISFs  and  to  coanunlcate 
the  operational  requirements  for  ISFs  to  personnel  Involved  In  the  acquisition  process.  These 
personnel  Include  operational  users  at  the  Air  Force  Mjor  commands  (MAJCOMs)  who  Initially  state 
training  requirements.  Simulator  Systems  Program  Office  (SlmSPO)  personnel  Involved  In  the  final 
specification  definition,  and  contracting  personnel  Involved  In  system  development. 


II.  BACKGROUND 

In  response  to  a  recognized  need  for  guidelines  that  might  be  applied  In  the  development  of 
specifications  for  the  Instructional  component  of  ATDs,  a  research  and  development  (RAD)  effort 
was  jointly  coordinated  by  the  Air  Force  HuMn  Resources  Laboratory,  Operations  Training  Division 
(AFHRL/OT) ,  Williams  AFB,  Arizona;  Headquarters  Military  Airlift  Coemund  (HQ  MAC),  Scott  AFB, 
Illinois;  and  Air  Force  Systems  Coamwnd,  Aeronautical  Systems  Division  (AFSC/ASD),  Wrlght- 
Patterson  AFB,  Ohio.  Although  this  RAD  effort  stemmed  from  the  development  of  the  perforMnce 
measurement  system  for  the  C-5  flight  simulator,  because  of  the  generic  nature  of  the  system 
architecture,  the  specifications  have  applicability  to  a  wide  variety  of  ATDs  from  Tactical  Air 
Command  (TAC),  Strategic  Air  ComMnd  (SAC),  and  Air  Training  ComMnd  (ATC).  The  formal 
requirements  for  this  RAO  were  Request  for  Personnel  Research  (RPR)  76-20,  Aircrew  Performance 
Measurement  System  (MAC/DOTF) ,  and  Technology  Need  (TN)  ASD-0509-82-63,  Flight  Simulator 
Performance  Metrics  (ASO/EN,  ASD/YWE).  This  advanced  development  effort  (PE  63751F)  was 
accomplished  as  part  of  AFHRL/OT's  training  effectiveness  plan,  under  work  unit  2359-01-06, 
Performance  Measurement  System  for  Aircrew  Training  Devices.  The  work  unit  was  validated  20 
December  1962,  approved  20  October  1983,  contracted  with  Loglcon  Inc.  on  1  May  1984,  and 
completed  30  September  1985.  The  products  of  this  RAD  effort  consist  of  the  following  technical 
reports  (TRs): 

1.  Easter,  A.W.,  Kryway,  J.T.,  Olson,  W.R.,  Peters.  S.M.,  Slemon,  G.K.,  A  Obermayer,  R.W. 
(1986a).  Development  of  Instructor  support  feature  guidelines  (AFHRL-TR-85-57U), 
A0-A168  308).  Williams  AFB,  AZ:  Operations  Training  Division,  Air  Force  Humh  Resources 
Laboratory. 


2.  Easter,  A.W.,  Kryway,  J.T.,  Olson,  W.R.,  Peters,  S.M.,  Slemon,  G.K.,  &  Obermayer,  R.W. 
(1986b).  Instructor  support  feature  guidelines  (AFHRL-TR-85-57(II),  A0-A168  285). 
Williams  AFB,  AZ:  Operations  Training  Division,  Air  Force  Human  Resources  Laboratory. 

The  RAO  that  led  to  the  development  of  the  ISF  Guidelines  is  documented  In 
AFHRL-TR-85-57U).  The  ISF  Guidelines,  which  provide  procedures  for  analyzing  training  and 
Instructor  support  requirements  to  specify  Instructor  support  systems,  provide  descriptions  of 
available  ISFs,  and  describe  current  system  capabilities,  are  contained  in  AFHRL-TR -85-57(11). 

Exploratory  development  work  (PE  62205F),  which  provided  the  data  base  for  development  of  the 
ISF  Guidelines,  was  conducted  under  work  unit  1123-03-58,  Flying  Training  Research  Support 
Services.  Surveys  were  conducted  (a)  to  document  the  utility  and  utilization  of  advanced 
Instructional  features  (AIFs)  in  ATDs,  (b)  to  compare  AIF  utility  and  utilization  In  replacement 
and  continuation  training  units,  and  (c)  to  provide  a  data  base  for  defining  requirements  for 
future  ATD  procurements  and  for  developing  subsequent  training  programs  using  ATDs.  The  results 
of  these  surveys,  documented  in  the  TRs  listed  below.  Indicated  that  most  Instructors  receive 
little  training  in  AIF  use  and  that  most  of  the  features  are  not  used  very  often. 

1.  Polzella,  D.J.  (1983).  Aircrew  training  devices:  Utility  and  utilization  of  advanced 

instructional  features  (Phase  I  -  Tactical  Air  Command)  (AFHRl-TR-83-22,  AD-A135  052). 
Williams  AFB,  AZ:  Operations  Training  Division,  Air  Force  Human  Resources  Laboratory. 

2.  Polzella,  D.J.  (1986).  Aircrew  training  devices:  Utility  and  utilization  of  advanced 

instructional  features  (Phase  11  -  Air  Training  Command.  Military  Airlift  Command,  and 
Strategic  Air  Command)  (AFHRL-TR-85-48,  AD-A166  726).  Williams  AFB,  AZ:  Operations 
Training  Division,  Air  Force  Human  Resources  Laboratory. 

3.  Polzella,  D.J.,  A  Hubbard,  D.C.  (1986).  Aircrew  training  devices:  Utility  and 

utilization  of  advanced  instructional  features  (Phase  III  -  Electronic  warfare  trainers) 
(AFHRL-TR-85-49,  AD-A167  922).  Williams  AFB,  AZ:  Operations  Training  Division,  Air 

Force  Human  Resources  Laboratory. 

4.  Polzella,  D.J.,  A  Hubbard,  D.C.  (In  press).  Aircrew  training  devices:  Utility  and 

utilization  of  advanced  instructional  features  (Phase  IV  -  Summary  Report) 

( AFHPL-TR-87-XX) .  Williams  AFB,  AZ:  Operations  Training  Division,  Air  Force  Human 

Resources  Laboratory.  This  report  is  currently  in  draft  review. 

Other  exploratory  development  work  at  AFHRL  (PE  62205F)  Involving  the  instructional  component 
of  ATDs  is  being  conducted  under  work  unit  1123-03-79,  Flying  Training  Research  Support,  and  will 
be  documented  in  the  following  report: 


1.  Warner,  H.D.  (In  press).  Instructor  operator  station  design  handbook  for  aircrew 
training  devices  (AFHRL-TR-87-XX( I ) ).  William  AFB,  AZ:  Operations  Training  Division. 
Air  Force  Human  Resources  Laboratory.  This  volume  is  currently  in  draft  review. 

2.  Charles,  J.  (In  press).  Instructor  operator  station  design  guide  (AFHRL -TR-87 -XX (II)). 
Williams  AFB,  AZ:  Operations  Training  Division,  Air  Force  Human  Resources  Laboratory. 
This  volume  is  currently  In  draft  review. 

volume  I,  the  I0S  Design  Handbook,  provides  the  human  factors/engineering- type  data  required 
to  develop  I0S  design  specifications.  Volume  II,  the  IOS  Design  Guide,  is  designed  to  assist  the 
user  In  determining  what  type  of  IOS  is  required,  based  on  the  type  of  ATD. 


These  volumes  are  designed  to  be  used  Independently  or  In  conjunction  with  the  ISF 
Guidelines.  The  overall  goal  Is  to  Improve  the  effectiveness  of  ATDs,  with  special  emphasis  on 
ensuring  that  the  Instructional  features  selected  address  the  training  requirements  and  are 
accurately  described  In  the  procurement  specifications. 


III.  DEVELOPMENT  COST 

Hork  unit  2359-01-06  was  formally  closed  out  on  18  February  1986.  Job  Order  Cost  Accounting 
System  (JOCAS)  data  Indicate  that  the  total  cost  of  the  ISF  Guidelines  effort  was  $449, 198,  of 
which  ASO  provided  $100,000. 


IV.  RESULTS 

Assessment  Methodology 

User  organizations  (l.e.,  organizations  Involved  In  defining  training  requirements.  In 
writing  procurement  specifications,  and  In  systems  development)  were  surveyed  from  August  to 
December  1986  to  determine  their  utilization  of  the  ISF  Guidelines.  Eighteen  RAD  Product 
Utilization  Assessment  surveys  were  sent  by  letter  to  14  organizations,  and  followed  up  by 
telephone  and/or  personal  visits.  A  copy  of  the  survey  Is  shown  In  Appendix  B.  Twelve  surveys 
were  returned  from  10  organizations.  A  listing  of  organizations  that  provided  utilization 
Information  Is  shown  In  Appendix  C.  The  responses  provided  were  Individual  opinion  and  are  not 
necessarily  representative  of  the  organization  with  which  the  Individual  Is  associated.  However, 
responses  are  documented  by  organization  as  a  matter  of  convenience. 


Implementation  Assessment 

The  ISF  Guidelines  document  was  designed  for  use  by  MAJCOM  requirements  personnel,  SlmSPO 
specification  writers,  ATD  users,  and  contractor  personnel.  When  It  was  published  In  May  1986, 
approximately  250  copies  were  printed.  In  addition  to  the  copies  sent  to  AFHRL  Primary 
Distribution  List  addressees,  approximately  10  copies  were  sent  to  ASD/YNE,  15  copies  to 
USAFTAWC,  and  50  copies  to  AFHRL /OT  for  special  distribution  to  user  organizations.  As  the 
primary  requirements  manager,  ASD/YNE  Is  the  responsible  office  for  ensuring  that  the  guidelines 
are  distributed  to  users.  This  organization  Is  also  listed  In  the  ISF  Guidelines  as  the  point  of 
contact  for  obtaining  additional  copies.  ASD/YNE  Indicated  that  at  the  completion  of  the 
contract,  they  sent  copies  of  the  Guidelines,  some  In  draft  form,  to  USAFTANC,  HQ  SAC/DOTP,  and 
seven  contractors  Involved  In  ATD  development.  As  of  January  1987,  AFHRL/OT  had  sent 
approximately  30  copies  to  users  upon  request,  and  AFHRL/XO  had  sent  10  copies  to  Individuals  who 
either  had  not  received  copies  from  ASD  or  were  requesting  additional  copies. 

The  amount  of  utilization  and  degree  of  usefulness  of  the  ISF  Guidelines  reported  varied 
considerably  among  the  Individuals  who  responded  to  the  survey.  This  Is  Illustrated  by  the 
response  frequencies  for  the  scaled  questions  shown  In  Figure  1. 

The  responses  Indicate  that  seven  of  the  twelve  Individuals  felt  that  the  ISF  Guidelines  were 
useful  and  were  being  used  at  least  moderately  within  their  organizations.  One  Individual  whe 
had  just  received  a  copy  reported  that  the  document  had  not  been  used  very  much  but  1nd1cate< 
that  It  would  likely  be  used  In  the  future.  Overall,  seven  organizations  (nine  Individuals)  hat 
very  positive  comments;  only  three  Individuals  (HQ  MAC/DOT,  USAFTAWC/TN,  and  McDonnell-Dougla; 
Helicopters)  had  unfavorable  comments. 


1.  To  what  extent  were  the  needs  of  your  organization  addressed  during  this  RAD  effort? 


1  2  3  4  5 


No  Very  Inadequately 

Neutral 

Adequately 

Very 

Response  Inadequately 

Adequately 

Frequency  1  2 

1 

6 

2 

2.  How  useful  Is  this  product  to  your  organization? 

1  2  3  4  5 


No  Not  at 

Not  Very 

Sufficiently 

Very 

Extremely 

Response  A1 1 

Useful 

Useful 

Useful 

Useful 

Frequency  1 

3 

1 

7 

3.  To  what  extent  Is  the  RAD  product  being  used  by  your  organization? 

1 

2 

3 

4 

5 

Not  at 

To  a  Little 

To  a  Moderate 

To  a  Great 

To  a  Very 

All 

Extent 

Extent 

Extent 

Great  Extent 

Frequency 

4 

3 

3 

2 

Figure  1.  Response  Frequencies  for  Utilization  Assessaent  Question. 

Four  organizations  who  responded  favorably  either  had  used  or  were  presently  using  the  ISF 
Guidelines  in  the  design  of  specific  ATDs,  as  Indicated  below: 

A-6F/F-14D  (Navy)  Operational  Flight  Trainer  (Boeing  Military  Airplane  Company) 

0130  Aircrew  Training  System  (Boeing  Military  Airplane  Company) 

F-15E  Weapon  System  Trainer  (Goodyear  Aerospace  Corporation) 

F-16  Operations  Flight  Trainer  (Singer-Link  Division) 

Modular  Instructor  Operator  Station  (Harris  Corporation) 

It  was  comonly  stated  (by  ASD/YWE  and  six  contractors)  that  the  ISF  Guidelines  docuaent  Is  a 
very  good  source  reference.  In  that  It  standardizes  definitions  and  applications  of  ISFs.  It 
also  provides  lessons  learned  froa  previous  applications,  which  Is  useful  In  avoiding  potential 
design  probleas  and  expenditure  of  tlae  and  aoney  on  research  required  to  deteralne  what  features 
to  lapleaent.  It  was  frequently  stated  that  standardization  of  the  ATD  teralnology  was  long 
overdue.  The  standardization  of  teras  facilitates  the  deteralnatlon  of  what  features  need  to  be 
Incorporated  In  ATDs  to  satisfy  the  training  requlreaents  specified  by  operational  users. 

One  negative  coaaent  (MAC/DOT)  was  that  the  guidelines  do  not  provide  adequate  supporting 
Inforaatlon  to  justify  the  costs  of  ISFs.  Provision  of  supporting  data  for  cost  justification 
purposes  was  not  an  objective  of  the  guidelines.  However,  as  part  of  their  overall  training 
effectiveness  plan,  AFHRL/OT  Is  currently  developing  software  aodules  that  will  address  the  cost 
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and  training  effectiveness  of  ISFs,  based  on  training  requirements.  Another  comment  by  MAC  was 
that  much  of  the  Information  contained  In  the  guidelines  is  common  knowledge.  In  discussing  the 
usefulness  of  the  ISO  Guidelines  as  a  reference  document,  most  of  the  other  organizations  seemed 
to  disagree.  As  an  example,  a  user  at  Reflectone  made  the  following  statement:  "Previously  this 
Information  was  only  available  through  many  years  of  experience  and  detailed  conversations  with 
the  users."  A  final  concern  expressed  by  MAC  was  that  the  ISF  Guidelines  may  be  overcome  by 
events  and  thus  become  no  longer  necessary.  Two  reasons  were  given:  First,  ATD  hardware 
requirements  are  no  longer  stated  In  MAC'S  Requests  for  Proposal  (RFPs);  only  the  training 
requirements  are  specified.  The  contractors  have  the  prerogative  to  decide  on  the  ATD 
configuration  as  long  as  the  device  proves  training  effective.  Second,  the  Air  Force  now 
contracts  out  weapon  system  training  programs;  therefore,  the  MAJCOMs  will  no  longer  require  the 
ISF  Guidelines  to  state  their  training  requirements.  AFHRL/OT's  reply  to  these  two  comments  Is 
as  follows:  First,  the  ISF  Guidelines  primarily  address  software  requirements;  only  rarely 
mentioned  Is  the  hardware  configuration  that  Is  required  to  support  the  software.  Second,  the 
ISF  Guidelines  would  still  be  of  use  to  the  contractors  who  develop  the  weapon  system  training 
programs. 

Another  criticism  (USAFTAHC/TN)  was  that  the  guidelines  have  a  limited  application.  Another 
Individual  (McDonnell -Douglas  Helicopters)  said  that  the  guidelines  have  a  fixed-wing  orientation 
and  deal  only  with  traditional  ISFs  at  the  undergraduate  pilot  training  level.  Followup  discus¬ 
sions  with  AFHRL/OT,  ASD/YWE,  and  several  contractors  failed  to  support  this  Interpretation.  The 
majority  opinion  was  that  the  ISF  guide  Is  generic  and  can  be  used  In  fixed-wing,  rotorcraft,  and 
even  maintenance  training  applications. 


V.  RECOMMENDATIONS 

The  organizations  contacted  for  this  utilization  assessment  made  various  recommendations 
either  as  ways  of  Improving  the  current  product  or  as  follow-on  RAD  that  would  be  beneficial. 
Their  recommendations  are  Indicated  below.  Reconmiendatlons  1  through  4  focus  on  ways  to  Improve 
the  current  guidelines;  5  through  9  pertain  to  follow-on  RAD. 

1.  Include  in  the  guidelines  the  rationale  for  Incorporating  advanced  training  features, 
their  advantages,  and  the  cost  savings  that  will  result  from  their  use. 

2.  Integrate  human  engineering  design  specifications  with  the  ISF  Guidelines. 

3.  Extend  the  scope  of  the  guidelines  to  tactical -level  applications.  Including  novel 
applications  to  multiship  use  and  features  specific  to  the  engineering  development  environment. 

4.  Incorporate  In  the  guidelines  more  sections  with  examples  of  actual  Instructor  consoles 
and  implementation  details. 

5.  Conduct  training  effectiveness  and  performance  measurement  RAD  on  the  design  requirements 
advocated  by  the  IOS  Design  Guide. 

6.  Investigate  design  errors  In  existing  weapon  system  trainers  and  publish  Information  on 
lessons  learned  and  how  the  problems  were  corrected. 

7.  Investigate  how  an  instructor/organization  might  best  use  artificial  Intelligence 
features. 


8.  Collect  field  data  on  actual  use  of  Instructional  features  and  user  acceptance 
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9.  Assess  commonalities  between  the  ISF  effort  and  ASO's  Modular  simulator  design  program. 

During  the  development  phase.  It  was  concluded  that  the  ISF  Guidelines  should  be  maintained 
as  a  dynamic,  "living"  document;  l.e.,  the  guidelines  must  be  continually  updated  to  effectively 
transition  lessons  learned  and  Incorporate  technology  advances.  However,  who  Is  responsible  for 
ensuring  that  the  document  Is  updated  on  a  timely  basis  and  the  Information  Is  correct?  How  will 
these  updates  be  made  on  a  continuing  basis?  Nhat  action  should  be  taken  to  correct  some  minor 
mistakes  contained  In  the  current  version  of  the  guidelines?  The  problem  of  keeping  the 
guidelines  current  was  partially  addressed  by  making  the  document  available  on  a  diskette  so  that 
each  using  organization  can  make  updates  or  corrections  whenever  appropriate.  However,  ASD 
reports  that  they  have  sent  only  two  or  three  diskettes  to  users.  From  various  discussions.  It 
seems  apparent  that  most  users  are  not  aware  of  the  diskette's  existence.  As  the  primary 
requirements  manager,  ASD/YWE  seems  the  appropriate  agency  for  Informing  users  of  the  diskette's 
availability  for  distribution  upon  request. 

At  the  beginning  of  this  utilization  assessment.  It  was  discovered  that  some  of  the  users  on 
the  ASD  distribution  list  had  not  received  their  copies  of  the  ISF  Guidelines.  Being  primarily 
responsible  for  distribution  of  the  guidelines,  how  should  ASD/YWE  ensure  that  the  users  are 
familiar  with  the  document?  MAC/DOT  recommended  that  the  guidelines  be  referenced  In  RFPs,  to 
ensure  that  all  organizations  responding  to  an  RFP  would  use  the  document.  In  addition  to  using 
the  ISF  Guidelines  Air  Force  wide,  one  contractor  recommended  that  the  other  services  look  at  the 
document  and  either  endorse  It  or  write  their  own. 


,  VI.  CONCLUSIONS 

The  stated  purpose  of  the  ISF  Guidelines  Is  to  promote  a  better  understanding  of  ISFs  and 
their  uses.  Throughout  the  document,  the  emphasis  Is  on  meeting  the  functional  needs  of  the 
Instructor  rather  than  the  application  of  state-of-the-art  technology.  Although  there  were  some 
;  differences  of  opinion,  the  majority  of  users  contacted  stressed  the  usefulness  of  the  guidelines 

I  as  a  reference  document  that  provides  definitions  of  ISFs  which  standardize  much  of  the 

terminology.  This  In  turn  enhances  communication  among  the  operational  users  at  the  MAJCOMs, 
SimSPO  personnel,  and  contractor  personnel.  The  ISF  definition  portion  of  the  guidelines  has 
been  incorporated  by  ASD  Into  the  following  military  standard;  MIL-S-8724,  Military 
!•  Specification,  Simulator  Flight,  31  December  1985.  Considering  the  primary  objective  of  the 

J  guidelines,  the  document's  usefulness  as  a  reference  source,  and  the  Incorporation  of  the  ISF 

•  definitions  Into  a  military  standard,  the  ISF  guide  is  judged  a  success. 

The  ISF  Guidelines  represent  only  part  of  the  total  documentation  needed  In  the  development 
and  acquisition  of  the  Instructional  component  of  an  ATD.  User  expectations  that  the  ISF 
Guidelines  would  provide  more  information  than  was  Intended  perhaps  account  for  many  of  the 
inadequacies  expressed,  such  as  limited  application  and  lack  of  cost  justification  data.  These 
perceived  Inadequacies  are  currently  being  addressed  In  follow-on  work  and  documentation.  The 
Instructor  Operator  Station  Design  Handbook  (draft)  will  provide  human  factors/engineering- type 

I  data  for  the  development  of  IOS  design  specifications,  the  Instructor  Operator  Station  Design 

Guide  (draft)  will  provide  functional  requirements  for  IOSs,  and  software  modules  currently  under 
development  will  provide  supporting  data  to  justify  the  cost  of  ISFs  In  relation  to  their 
training  effectiveness. 
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APPENDIX  A:  INSTRUCTIONAL  SUPPORT  FEATURES 


Pre-Training  Requirements 

Instructor  Training  Function 
ISF:  Tutorials 

Briefing  Function 

ISF:  Briefing  Utilities 

Training  Requirements 

Control  Function 

ISFs:  Scenario  Control 

Initial  Conditions 

Real-Tine  Simulation  Variables  Control 

Malfunction  Control 

Reposition 

Monitor  Function 

ISFs:  I  OS  Display  Control  and  Formatting 
Procedures  Monitoring 

Instruct  Function 
ISFs:  Freeze 

Simulator  Record/Replay 
Automated  Simulator  Demonstration 

Evaluation  Function 

ISF:  Automated  Performance  Measurement 

Post-Training  Requirements 

Debriefing  Function 

ISFs:  Hardcopy/Printout 

Remote  Graphics  Replay 

Recording  Function 

ISF:  Data  Storage  and  Analysis 


i 
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APPENDIX  B:  RiO  PRODUCT  UTILIZATION  ASSESSMENT  SURVEY 


AIR  FORCE  HUMAN  RESOURCES  LABORATORY 
Air  Force  Systems  Command 
Brooks  Air  Force  Base,  Texas  78235-5601 


Note:  For  each  scaled  question,  please  circle  the  appropriate  number  and  comment  further  as  you 
desire.  If  necessary,  attach  additional  page  ir  your  comments. 


1.  To  what  extent  were  the  needs  of  your  organization  addressed  during  this 
R&D  effort? 


Very 

Inadequately 


2 

i 


i 

Inadequately 


Neutral 


Adequately 


5 

i 


i 

Very 

Adequately 


Comments:  (Please  be  as  specific  as  you  can.) 


How  useful  is  this  product  to  your  organization? 


2 

i 


i 

Not  Very 
Useful 


Sufficiently 

Useful 


5 

i 


i 

Extremely 

Useful 


Comments:  (Please  be  as  specific  as  you  can.) 


’AoZcixl*. 


3.  To  what  extent  is  the  R&D  product  being  used  by  your  organization? 


1 

1 

2 

1 

3 

1 

4 

i 

5 

i 

1 

Not  at 

1 

To  a 

1 

To  a 

1 

To  a 

1 

To  a 

All 

Little 

Moderate 

Great 

Very  Great 

Extent 

Extent 

Extent 

Extent 

Comments;  (If  the  product  is  not  being  used,  please  explain  why  this  is  the  case.  If 
it  is  used,  please  specify  any  concrete  benefits,  e.g.,  cost  savings,  increased 
operational  effectiveness,  improved  force  structure.  Also,  if  possible,  please 
include  the  time  required  for  implementation,  the  cost  of  implementation  and 
any  barriers  to  implementation/utilization.) 


4. 


Please  identify  (name/ofHce  symbol)  other  organizations  that  currently  use  this 
product. 

Unknown 


5.  Please  specify  any  additional  possible  uses  for  this  product. 


Unknown 


6. 


Please  identify  (naine/office  symbol)  other  organizations  that  could  possibly  use 
this  product. 

Unknown 


7. 


What  follow-on  R&D  would 


your  organization  consider  potentially  beneficial? 


Unknown 


8.  Please  comment  additionally  on  this  product  ( relevance ,  timeliness,  clarity,  completeness, 
etc.). 


9.  Please  complete  the  following  information: 

ORGANIZATION/OFFICE  SYMBOL:  _ 

NAME:  _  RANK  or  GRADE:  _ _ 

Person  completing  survey 

POSITION:  _ 

AUTOVON/COMMERCIAL  PHONE  NUMBER:  _ 

ADDRESS:  _ 

Street  or  P  0.  Bo* 

City  nr  Bane  Slite  Zip 
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THANK  YOU  FOR  PROVIDING  THIS  INFORMATION. 


AUTHORITY:  10  U.S.C.  8012  Secretary  of  the  Air  Force,  Powers,  Duties, 
Delegation  by  Compensation. 

PRINCIPAL  PURPOSE:  The  information  provided  will  be  used  solely  for  Air  Force 
management  purposes  related  to  R&D  utilization. 

ROUTINE  USES:  To  provide  feedback  to  the  Laboratory  on  the  utilization  of  R&D 
products. 

DISCLOSURE  IS  VOLUNTARY 


—  CoW  twre.  tuple,  uvt  m*\l - 


6570  ABC/DAA 
BROOKS  AFB.  TX  78235 


OFFICIAL  BUSINESS 
PENALTY  FOR  PRfVATE  USE.  *300 


NO  POSTAGE 
NECESSARY 
IF  MAILED 
IN  THE 

l  NfTED  STATES 


BUSINESS  REPLY  MAIL 

F1HST  CLASS  PERMIT  NO.  73236  WASH  DC 
postack  will  nr  paid  by  ho  i  nitto  states  aim  roaci 


PLANS  \M>  OPER  ATIONS  OKEIC  E 

AIR  FORC  E  III  MAN  RESOURCES  LABOR  MORA 
BROOKS  AIR  FORCE  BASE 
SAN  ANTONIO.  T\  782:L»-.V»0 1 


APPEWOIX  C:  ORGANIZATIONS  PROVIDING  UTILIV'T10N  INFORMATION 


Requlrem tnts  Organizations 

ASD/YWEE:  Aeronautic*!  Systems  Division,  Deputy  for  Simulators,  Training  System 

Engineering,  Engineering  Branch 

HQ  MAC/DOT:  Headquarters  Military  Airlift  CoaMnd,  Directorate  of  Training 
Additional  Organizations 

USAFTAHC/TN:  United  States  Air  Force  Tactical  Air  Warfare  Center, 

OCS/AIrcrew  Training  Devices 

Boeing  Military  Airplane  Company,  Simulation  and  Training  Systems,  Huntsville  At 
Cubic  Defense  Systems,  San  Diego  CA 
Goodyear  Aerospace  Corporation,  Akron  OH 
Harris  Corporation,  Winter  Park  FL 

McDonnell -Douglas  Helicopters,  Engineering  and  Training  Simulation  Department,  Mesa  AZ 
Reflectone.  Displays  System  Engineering  Department.  Tampa  FL 
Singer-Link  Division,  Houston  TX 
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